ninjamiefandomcom-20200214-history
URL Filtering
The''' PAN-DB URL filtering database is a URL database that is maintained by Palo Alto Networks and offers an alternative to the BrightCloud Database. *''Cloud Service'', URL Management Plane Cache, and URL Dataplace Cache are the components used to support and process the PAN-DB URL Lookup process. '''URL Categorization resolution process flow: PAN DB: When a user attempts to access a URL and the URL category needs to be determined, the firewall will compare the URL with the following components until a match has been found: #Block list of the matching URL profile #Allow list of the matching URL profile #Custom categories that have been defined #DP URL cache #MP URL cache #Cloud systems - goes to the cloud. No need for the disk and the dataplane has a larger cache. 'Cloud Service': *implemented using Amazon Web Services (AWS) and allows PAN devices to download a seed database, which is then used for the initial population of the device cache. *The Cloud holds the entire PAN-DB and is updated as new URL categories are identified. *Lookup operation on the cloud database is a best-match operation. *The PAN-DB is marked with a database version (for updating purposes) and a protocol version (for compatibilty purposes). *The cloud service supports an automated mechanism to update the firewall's local URL database if the version does not match. Each time the firewall queries the cloud servers for URL lookups, it will also check for critical updates. **If there have been no queries to the cloud servers for more than 30 minutes, the firewall will check for updates at that time. *The cloud service is accessed through secured access only (SSL) 'BrightCloud:' *5,000 to 10,000 in the data plane cache *Stores 1 million last URLs in Management Plane. *The local disk file is the difference between PAN-DB and the BrightCloud. 'URL Management Plane (MP) Cache': *A customized dynamic URL database is stored in the management plane. *A seed database is downloaded from one of the PAN-DB cloud servers to initially populate the local caches. Each regional seed database contains the top URLs from that region. The size of the seed database (number of URL entries) depends on the device platform as cache sizes vary per device. *The PAN-DB version in the cloud and the version installed on the firewall may differ, so the version on the firewall will have one of the follow states: **''Good'' = The version on the firewall is the same as the version in the cloud. **''Out-of-Date'' = The version on the firewall is out of date and the cloud does not have the ability to sync the firewall's URL database. In this case the connection with the cloud will be blocked until the administrator performs a manual update by re-downloading a seed database. *A firewall may also start with an empty PAN-DB seed. This could be due to a corrputed seed, or the administrator may have deleted the existing seed from the MP. In this case the cloud will automatically sync the PAN-DB to the MP. *The URL MP cache is automatically written to the firewall's local drive every 8 hours, or when the cloud upgrades the URL database verison on the firewall. The PAN-DB is written in a compressed version on the cache image. The header of the file includes the database version and the device-cloud protocol version (to ensure software copatibility). After rebooting, the file that was saved to the local drive will be loaded to the MP. *IF the Cache is FULL: a Least Recently Used (LRU) mechanism is implemented. The URLs that have been accessed the least will be replaced by the new URLs. *Entries in the URL MP cache expire after a certain period of time and is set by the PAN-DB core per URL which cannot be changed by the administrator: **Never, 7 days, 1 day, 1 hour, or 30 minutes. 'URL Database (DP) Cache': *A customized dynamic URL database that is stored in the dataplane (DP). *The URL DP cache is cleared at each firewall reboot. *IF the Cache is FULL: a Least Recently Used (LRU) mechanism is implemented. The URLs that have been accessed the least will be replaced by the new URLs. *Entries in the URL DP cache expire after a certain period of time and cannot be changed by the administrator: **Never, 7 days, 1 day, 1 hour, or 30 minutes. If a URL query in the URL DP cache matches an entry that is expired, the URL DP cache responds with the expired category, but also sends a URL categorization query to the MP. *This avoids unnecessary delays in the DP, assuming that the frequency of changing categories is low. *Similarily, in the URL MP Cache, if a URL categorization query from the DP matches an entry that expired in the MP, the MP responds back to the DP with the expired category and will also send a URL categorization request to the cloud service. Upon getting the response from the cloud, it will resend the updated response to the DP. 'Incorrect Categorization': #Verify if the category in the dataplane (DP) is incorrect: ##> show running url'' '' #Verify if the category in the Mangement Plane (MP) or Cloud is incorrect by running: ##>'' test url '' ###'If the URL stored in the MP cache has the correct category, EX: "cnn.com news ('Base db') expire in 0 seconds", remove the URL from the URL DP cache by running: ####> ''clear url-cache url '' ###The next time the device will request the category of the URL, the request will be forwarded to the MP #Verify if the category in the cloud is incorrect by running ''test url '' ##IF the output is "cnn.com news ('cloud DB')''", remove the URL from the DP/MP cache by using the following command: ###Delete URL from the DP: > 'clear url-cache url ''' ###Delete URL from the MP: > ''delete url-database url '' ###The next time the device will ask for the category of the URL, the request will be forwarded to the MP and then to the cloud. #Submit a change request from the web interface by going to the URL log and select the log entry with the URL you would like to change. ##Click the request categorization change link and follow the instructions provided. Commands >'show url-cloud status' PAN-DB URL Filtering License : valid Current cloud server : s0200 Cloud connection : ' connected' URL database version - device : 2014.03.24.220 URL database version - cloud : 2014.03.24.224 ( last update time 2014/03/24 12:23:58 ) URL database status : good URL protocol version - device : pan/0.0.2 URL protocol version - cloud : pan/0.0.2 Protocol compatibility status : compatible If URL database status is Out-of-Date: Download a new seed by running the command: > request url-filtering download paloaltonetworks region North-America PAN-DB update initiated *The connection from the firewall to the URL cloud is blocked. Usually occurs when the URL database on the firewall is too old (version difference is more than 3 months) and the cloud cannot update the firewall automatically. *Re-downloading the inital seed database from the cloud will automatically re-activate the PAN-DB. If URL protocol version is not compatible: Upgrade the PAN-OS software version.